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Abstract The purpose of this article is to describe the 
practice-based coaching model used in BEST in CLASS, a 
Tier-2 classroom-based intervention comprised of evi- 
dence-based instructional practices designed to prevent and 
ameliorate the chronic problem behaviors of young chil- 
dren at risk for the development of emotional/behavioral 
disorders. Following a description of the model, data from 
year two of an ongoing 4-year randomized control trial are 
presented that describe (a) the amount (i.e., dosage) of 
coaching teachers received during BEST in CLASS 
implementation, (b) the integrity with which coaches 
implemented the BEST in CLASS coaching model, and 
(c) subsequent teacher implementation of the BEST in 
CLASS strategies. Forty-eight (23 BEST in CLASS; 25 
comparison) teachers and ten coaches participated in this 
descriptive study. Data indicate that following coaches’ 
observations, teachers received approximately 30 min of 
practice-based coaching during coaching meetings each 
week of implementation, and integrity data indicate that 
coaches implemented the critical coaching skills during the 
coaching meetings with integrity. Adherence data indicate 
that teachers in the treatment group increased their exten- 
siveness of the use of BEST in CLASS practices at both 
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post-treatment and 1 -month follow-up compared to com- 
parison teachers; competence data indicated that teachers 
in the treatment group increased the quality of delivery of 
practices at post-treatment compared to comparison 
teachers. Implications of these findings for both future 
research and practice-based coaching implementation are 
discussed. 

Keywords Early intervention ■ Practice-based coaching ■ 
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Introduction 

Optimizing young children’s learning through high-quality 
early childhood programming is gaining much needed 
national attention. Recently, federal agencies (US Depart- 
ment of Health and Human Services, 2012) and profes- 
sional organizations (e.g., National Association for the 
Education of Young Children) have emphasized the short - 
and long-term benefits for children when they have an 
opportunity to access high-quality learning environments 
during their early years. Along with the emphasis on pro- 
viding high-quality early childhood environments for 
young children is a parallel emphasis on providing high- 
quality professional development opportunities for early 
childhood teachers to assure that they have the knowledge 
and skills needed for creating such environments (e.g., see 
U. S. Department of Health and Human Services, 2012). 

At the classroom level, a high-quality learning envi- 
ronment includes a number of important instructional ele- 
ments, such as providing a safe classroom environment that 
is responsive to children’s social, emotional, and pre-aca- 
demic developmental needs and the use of organizational, 
instructional, and classroom management strategies that 
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promote child engagement and learning (e.g., see Abry, 
Rimm- Kaufman, Larsen, & Brewer, 2013; Curby, Rimm- 
Kaufman, & Abry, 2013; Williford, Vick Whittaker, Viti- 
ello, & Downer, 2013). To ensure that these elements are in 
place within a classroom, early childhood teachers need to 
have both a solid understanding of these critical elements 
and be proficient in using them across a variety of contexts 
with all children in their classrooms. 

Although many early childhood teachers do possess 
knowledge of these key elements and incorporate them into 
their classrooms to varying degrees, classrooms are 
dynamic environments and children’s learning in these 
environments is influenced by a number of other factors, 
including the individual characteristics of the children 
(Bradley & Corwyn, 2002; Hetzner, Johnson, & Brooks- 
Gunn, 2011; Patterson, Reid, & Eddy, 2002; Yoshikawa, 
Aber, & Beardslee, 2012). Even in the most high-quality 
early childhood classrooms, teachers may be challenged to 
provide high-quality environments for all children, espe- 
cially those who need more intensive instruction due to 
learning and behavioral challenges. To create high-quality 
learning environments to meet these children’s needs, early 
childhood teachers often need more intensive and focused 
professional development activities which can strengthen 
their knowledge and skills in learning how to implement 
evidence-based practices with individual children in their 
classrooms (Becker & Domitrovich, 2011; Joyce & 
Showers, 2002; Snyder & Wolfe, 2008; Trivette, Dunst, 
Hamby, & O’Herin, 2009). 

Professional Development for Individualizing 
Instruction 

When creating high-quality environments for individual 
children with learning and behavioral challenges, “one size 
does not fit all.” Typically, children with learning and 
behavioral problems need a more intensive and individu- 
alized approach to instruction. Likewise, teachers who 
work with these children are likely to need a more intensive 
and individualized approach to professional development 
(Conroy, Alter, Boyd, & Bettini, 2014). Teachers may have 
a general knowledge of effective strategies and practices 
that can be used to support children’s learning, but they 
often indicate the need for more professional development 
and support in acquiring the skills needed to implement 
these individualized practices with specific children in their 
classrooms (Hemmeter, Corso, & Cheatham, 2006). 

As indicated by the National Professional Development 
Center on Inclusion (2008), high-quality professional 
development is a transactional process and should provide 
teachers with the structure and support for learning and 
applying knowledge and skills within their classrooms. 
More importantly, high-quality professional development 


is linked to high-quality teaching and positive outcomes for 
children (for a discussion, see Snyder, Denney, Pasia, 
Rakap, & Crowe, 2011). Practice-based coaching is one 
type of professional development that is more individual- 
ized and is designed to support teachers’ implementation of 
effective practices within their classrooms (Snyder, Hem- 
meter, et al., 2011; Snyder, Denney et al., 2011; Snyder & 
Wolfe, 2008). 

Originally developed by Snyder (2007), practice-based 
coaching is a collaborative professional development 
model in which a teacher and a coach work together to 
assess the classroom needs, plan and implement the use of 
select instructional practices within the classroom, and 
evaluate the effectiveness of these practices on desired 
teacher and child outcomes (for a discussion, see Snyder, 
Hemmeter et al., 2011; Snyder et al., 2012; Snyder & 
Wolfe, 2008). Typically, within a practice-based coaching 
framework, a three-step process occurs. First, the teacher 
and the coach conduct a needs’ assessment to identify goals 
and instructional practices to help accomplish those goals. 
This initial step is followed by the development of an 
action plan that outlines how these practices will be 
implemented within the classroom. Next, the coach con- 
ducts a direct observation of the teacher’s implementation 
and provides the teacher with performance feedback and 
support, while the teacher reflects on his or her use of the 
practice and responds to feedback. These steps are repeated 
over time until the desired teacher and child goals and 
outcomes have been met. As suggested by Joyce and 
Showers (2002) when used in combination with other 
professional development activities, such as modeling and 
practice, teachers who receive coaching have a higher 
likelihood of using the practices within their classrooms. 

A number of studies (e.g., Conroy, Sutherland, Vo, Carr, 
& Ogston, 2014; Conroy, Sutherland, Algina, et al., 2014; 
Fox, Hemmeter, Snyder, Binder, & Clarke, 2011; Hem- 
meter, Snyder, Kinder, & Artman, 2011) have employed a 
practice-based coaching model to support teachers’ use of 
effective practices to improve children’s social, emotional, 
and behavioral outcomes. For example, in a recent study 
Fox et al. (2011) examined the relationship between pro- 
fessional development training, including coaching, and 
teachers’ implementation of promotion and prevention 
practices. They found that the dosage of coaching was an 
important factor associated with high levels of teachers’ 
implementation of selected practices. In addition, Hem- 
meter et al. (2011) found that data-based performance 
feedback was an important component in changing teacher 
behavior. 

One key finding from these studies is that when pro- 
fessional development models include practice-based 
coaching, teachers implement targeted practices within 
their classrooms with integrity and child outcomes 


*£) Springer 


School Mental Health 


improve. As discussed earlier, both teacher implementation 
integrity and child outcomes are important because they 
emphasize the connection between high-quality profes- 
sional development (i.e., practice-based coaching), high 
levels of integrity of implementation of intervention prac- 
tices by teachers, and positive child outcomes. Although 
the relationship between integrity and child outcomes has 
been highlighted in the literature (see Durlak, 2010), the 
field continues to struggle with understanding what com- 
ponents of practice-based coaching are important in order 
to assure that teachers have the necessary knowledge and 
skills so practices can be delivered with high levels of 
integrity. An initial step of this process is describing the 
key features and components of practice-based coaching. 
Therefore, the purpose of this article is to describe the 
practice-based coaching model used in BEST in CLASS, a 
Tier-2 classroom-based intervention designed to prevent 
and ameliorate the chronic problem behaviors of young 
children at risk for the development of emotional/behav- 
ioral disorders, and examine (a) the amount (i.e., dosage) of 
coaching teachers received during BEST in CLASS 
implementation, (b) the integrity with which coaches 
implemented the BEST in CLASS coaching model, and 
(c) treatment acceptability using data from teachers in the 
treatment condition in year two of a randomized control 
trial (RCT) that is currently in its fourth and final year. In 
addition, teacher implementation of BEST in CLASS 
strategies will be examined using data from both treatment 
and comparison conditions in year 2 of the RCT. While 
other investigations report child and teacher outcomes 
related to the BEST in CLASS model (see Conroy, Suth- 
erland, Vo, et al., 2014; Conroy, Sutherland, Algina, et al., 
2014; Sutherland, McLeod, Conroy, Abrams, & Smith, 
2014; Sutherland, Conroy, Vo, Abrams, & Ogston, 2013), 
this is the first article to present descriptive data on the 
practice-based coaching component of the model and as 
such provides important information related to the use of 
practice-based coaching and teacher implementation of 
identified practices. 

Methods 

BEST in CLASS Practice-Based Coaching 

BEST in CLASS is comprised of evidence-based instruc- 
tional practices designed to prevent and ameliorate problem 
behaviors, enhance positive teacher-child interactions, and 
promote engagement and appropriate behaviors in young 
children at risk for emotional and behavioral disorders 
(EBD). BEST in CLASS is delivered by teachers to focal 
children in the classroom who are at risk of EBD; therefore, 
it is important that teachers gain both the knowledge and 


skills for implementing the BEST in CLASS practices with 
high levels of integrity. This is accomplished through a 
targeted professional development program comprised of a 
1-day didactic group workshop followed by 14 weeks of 
individualized practice-based coaching. During the 1-day 
workshop and throughout practice-based coaching, teachers 
and coaches use a standardized manual to discuss and plan 
for the use of specific practices associated with BEST in 
CLASS, namely Rules, Precorrection, Behavior Specific 
Praise, Opportunities to Respond, Corrective Feedback, and 
Instructive Feedback. For specific information about these 
practices and their use within BEST in CLASS, see Conroy, 
Sutherland, Vo, et al. (2014). 

BEST in CLASS coaches and teachers utilizes the 
practice-based coaching model originally developed by 
Snyder (2007) that is sustained, collaborative, and cyclical. 
Coaches are trained to (a) demonstrate mastery of the 
BEST in CLASS practices; (b) demonstrate competence in 
providing quality practice-based coaching and its applica- 
tion within the BEST in CLASS model; (c) demonstrate 
collaborative communication skills including reflective 
listening, goal setting, self-reflection, and related skills; and 
(d) explain to teachers how each of the BEST in CLASS 
data collection tools can be used to evaluate progress 
toward meeting specific goals. Collaborative communica- 
tion skills are stressed during training of coaches to facil- 
itate teacher ownership of and partnership in the process. 
Following training, all coaches demonstrate mastery in the 
essential coaching skills by completing a mock coaching 
meeting which is video-recorded and coded by a reliable 
observer using a coaching integrity form discussed below. 
Mastery is attained when a coach demonstrates all of the 
vital components of the coaching model (below) within the 
context of the coaching meeting and when he/she demon- 
strates the required communication and collaboration 
skills. Participating teachers are initially trained through a 
1-day didactic group workshop to lay the knowledge 
foundation for practice-based coaching. At the workshop, 
teachers receive their BEST in CLASS teacher manual and 
participate in hands-on activities, role-play scenarios, and 
view modeled and video examples of implementation of 
each BEST in CLASS practice within a classroom. 

Coaching begins approximately 1 week after the work- 
shop and occurs for 14 consecutive weeks. Each week can be 
divided into three parts; the classroom observation, the 
coaching meeting between the teacher and coach that occurs 
later on the same day, and the practice and reflection time that 
occurs in between observation/coaching meeting days. 
Coaching within BEST in CLASS follows the cycle illus- 
trated in Fig. 1 and includes six distinct stages. For the 
purpose of this paper, they are organized numerically. In 
practice, one stage flows into another as the process repeats 
itself weekly. Stage 1 : facilitated instruction of new skill and 
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Fig. 1 Stages of the BEST in CLASS coaching model 


Stage 2: shared goals and action planning both occur during 
the weekly coaching meeting. Stage 1 begins with a brief 
assessment of the teacher’s understanding of the targeted 
BEST in CLASS practice. Facilitated instruction is scaf- 
folded from that point to ensure the teacher has a clear 
understanding of the definition and quality indicators of 
implementing the practice. The action plan includes the 
definition and examples of its use, which are determined by 
the coach and teacher collaboratively, and a linking state- 
ment illustrating how the practice can be used in combination 
with other BEST in CLASS practices. In Stage 2, the coach 
and teacher work together to identify problem behaviors 
displayed by focal children and the settings in which they 
occur. They then create an individualized goal utilizing the 
targeted BEST in CLASS practice to address the specific 
behaviors and settings identified. The action plan includes a 
goal statement addressing quality and quantity for the tea- 
cher’s behaviors, expected child outcomes, how information 
will be shared with caregivers, when and where the strategy 
will be used, what the teacher will do to accomplish the goal, 
how the teacher will know if he/she has met the goal, and the 
supports that will be provided by the coach. Each week, the 
coach and teacher select either modeling or prompting as a 
support in addition to other material and informational sup- 
ports such as encouragement and reminders. While planned 
during the coaching meeting, the modeling and prompting 
may not occur within the classroom until a later time when 
the coach is in the classroom. During modeling, coaches may 
role-play with teachers or may interact briefly (approxi- 
mately 5 min) with children to model the use of a predeter- 
mined strategy within a specific context. During prompting, 
the coach prompts the teacher (verbally or using a gestural 
cue) during instruction to signal the teacher to use a prede- 
termined strategy. 


Stage 3: guided practice occurs during the week as the 
teacher practices the steps involved in the plan. The coach 
checks in with the teacher throughout the week via email, 
phone, and/or site visits to offer additional information 
about the targeted BEST in CLASS practice of the week 
and to provide any supports agreed upon during the action 
planning stage. During this time, both coaches and teachers 
engage in Stage 4: reflection. Teachers are encouraged to 
self-reflect on their utilization of the targeted BEST in 
CLASS practice and communicate any important insights 
to the coach using a Teacher Weekly Self-Reflection form. 
The coach reflects on his/her most recent observation and 
coaching meeting to ensure that all stages of the BEST in 
CLASS coaching process were completed with integrity. 
This is done as coaches self-reflect while completing the 
BEST in CLASS Coaching Integrity form (see below). 

Finally, Stage 5: direct, focused observation and Stage 
6: reflection and feedback occur in the teacher’s classroom 
during the next weekly visit. Coaches begin with a brief 
period of modeling and/or prompting of a specific strategy 
that was pre-arranged during the coaching meeting that 
occurred the previous week. Coaches then collect direct 
observation data utilizing the Teacher-Child Interaction 
Direct Observation System (TCIDOS; Sutherland, Conroy, 
et ah, 2013) as well as video and anecdotal notes to share 
with teachers. During this time, coaches do not interact 
with teachers or children. Rather, their focus is solely on 
observing the teacher and focal children interacting within 
a naturally occurring instructional situation. The final stage 
begins between the observation and coaching meeting and 
concludes with the sharing of the reflection and provision 
of feedback at the beginning of the coaching meeting. 
During this stage, the teacher reviews the appropriate 
weekly self-assessment form, while the coach uses this 
time to prepare feedback to be shared with the teacher 
during the coaching meeting. The coach and teacher 
(individually and together) attempt to identify what was 
effective and what barriers exist in order to refine the 
implementation of BEST in CLASS strategies for the tea- 
cher within the context of his/her classroom. This stage 
concludes with the teacher determining if the weekly goal 
has been met. If it has, the coach and teacher collabora- 
tively select a new goal. If the goal was not met, the coach 
and teacher discuss the reason(s) why and create a new 
goal and action plan including steps the teacher and coach 
will take to address any identified barriers. Upon conclu- 
sion of Stage 6, the entire process is repeated for 14 weeks 
with new practices or refinement of practices occurring 
during each week. All teachers who complete the 14 weeks 
of coaching work with coaches to focus on each BEST in 
CLASS strategy for 2 weeks followed by two concluding 
weeks focusing on linking all the BEST in CLASS prac- 
tices together. 
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Setting 

The current study took place in two southeastern states in 
federally or state-funded early childhood programs (e.g.. 
Head Start, Title I, state-funded prekindergarten) serving 
children aged 3-5 years old. Most students served in these 
programs were minorities and received free or reduced 
lunch. Since BEST in CLASS is an intervention added to 
the existing classroom curriculum, both the treatment and 
comparison classrooms continued with their ongoing 
"business as usual” instructional approaches. Upon post- 
test, we surveyed all treatment and comparison teachers to 
identify any other formal social, emotional, and behavioral 
interventions that were implemented throughout the year, 
and no differences were found between the two conditions. 

All of the classrooms in this study were housed in early 
childhood centers or within elementary schools. In one 
state, all early childhood classrooms were located within 
two Head Start programs. One Head Start was adminis- 
tered by a local school district, and the other was 
administered by a community-based early childhood 
agency. In the second state, classrooms were located 
within 12 elementary schools and four early childhood 
centers administered through three local school districts. 
Between 1 and 5, teachers participated from each school 
or early childhood center depending upon the number of 
classrooms available to participate within that school or 
center, with an average of 1.66 teachers per school/pro- 
gram. BEST in CLASS coaches worked with between 1 
and 4 teachers (M = 2.2) depending upon their position 
and assigned responsibilities. 

Participants 

Coaches 

Ten coaches implemented the BEST in CLASS coaching 
model. Nine were females, and one was male (age range 
25-65 years old). Seven coaches were Caucasian women, 
while the other three coaches were Hispanic. Five coaches 
were graduate students seeking a master’s or doctoral 
degree in education or a related field (School Psychology, 
Special Education, Counselor Education) and four were 
former teachers. One coach held a doctoral degree in 
Special Education. 

Teachers 

Teacher participants were first contacted about potential 
participation in the beginning of the school year. Teachers 
who consented to participate were contacted again 
approximately 1 month into the school year to help identify 
children in their classrooms who were potentially at risk for 


Table 1 Teacher demographics 


Item 

BEST in 
CLASS 

Comparison 

Total 

Number of teachers 

23 

25 

48 

Age range (%) (years) 




18-25 

8.7 

12.0 

10.4 

26-35 

26.1 

16.0 

20.8 

36 — 45 

21.7 

24.0 

22.9 

46-55 

17.4 

16.0 

16.7 

55+ 

26.1 

32.0 

29.2 

Female (%) 

100 

96 

97.9 

Race (%) 




Caucasian 

69.6 

40.0 

54.2 

African American 

26.1 

52.0 

39.6 

Hispanic 

0.0 

8.0 

4.2 

Other 

4.3 

0.0 

2.1 

Average years teaching 

10.3 (9.5) 

15.3 (11.6) 

12.8 (10.8) 

experience (SD) 




Highest level of education (%) 



Master’s degree 

21.7 

48.0 

35.4 

Bachelor’s degree 

39.1 

28.0 


Associate’s degree 

39.1 

24.0 



EBD. Up to three children in each class were screened for 
eligibility in the study after informed consent was obtained 
from their caregivers. Fifty-one teachers were randomly 
assigned to condition (BEST in CLASS or comparison) in 
year 2 of the parent study, and after attrition, a total of 48 
teachers completed the study with 23 teachers assigned to 
the BEST in CLASS treatment condition and 25 teachers 
assigned to the comparison condition. See Table 1 for 
further detail on the teacher demographics. 

Measures 

Several measures were used to obtain descriptive data on 
the practice-based coaching model used within the BEST 
in CLASS intervention. These included measures of 
coaching dosage and coaching integrity. 

Coaching Dosage 

Coaches’ self-reported coaching dosage following each 
weekly observation and coaching meeting with each par- 
ticipating teacher. Dosage data included the duration of the 
coaching meeting and the frequency of optional follow-ups 
of three types (email, phone, site visit). Averages of 
coaching dosage per teacher were calculated using the 
duration (in minutes) of coaching meetings divided by the 
total number of coaching meetings held. While the 
coaching meetings were designed to last approximately 
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Table 2 Coaching integrity procedural items 

Item 

1 . Completes observation and data collection according to specific 
protocols 

2. Provides opportunities for teacher self-assessment and 
facilitates teacher self-reflection 

3. Collects, prepares, and communicates video, anecdotal, and 
graphical data, including varied relevant examples of strategies 

4. Reviews goals, feedback, and teacher self-assessment to help 
teacher synthesize information 

5. Discusses and assists with home-school communication 

6. Collaboratively begins goal and action planning for following 
cycle 

7. Goal and action plan relate to specified BEST in CLASS 
strategy 

8. Goal and action plan are specific, measurable, and reasonable 

9. Goal and action plan includes quantity and quality components 

10. Supports teacher to select appropriate evaluation tools based 
on goal 

11. All components of goal and action plan are fully completed 


30 min each week, they varied in length from 10 to 60 min 
depending on the teachers’ needs, the amount of time 
needed to discuss the practice and develop the action plan, 
and unusual circumstances. For example, some teachers 
required additional support during some coaching meet- 
ings, which lead to longer meetings. Other teachers had 
emergencies come up resulting in truncated meeting times 
which were followed by additional email and/or phone 
follow-up. 

Coaching Integrity 

Coaching integrity data included coach self-reports as well 
as direct observations. Self-report data included a checklist 
of coaching components (e.g., modeling and/or prompting; 
observational data collection; videotaping the teachers’ 
implementation of practices during the selected activity; 
completing anecdotal notes). Trained observers at each 
research site collected observational data using the BEST 
in CLASS Coaching Integrity form. The form was divided 
into two categories: (1) collaboration competencies and (2) 
coaching procedures. Collaboration competencies include 
rapport building, communication skills, and collaborative 
planning, and were rated on a three-point Likert-type scale 
(i.e., 1 = not at all ; 2 = somewhat ; and 3 = extensive). 
Eleven BEST in CLASS coaching procedures (see Table 2) 
were recorded as present or absent (yes/no). Coaching 
sessions were coded either live or from video recordings. 

Prior to coding, coders at each site were trained on the 
procedures outlined in the Coaching Integrity Manual. 
Training included memorizing and discussing the coding 


definitions as outlined in the BEST in CLASS Coaching 
Integrity Manual and then practicing by coding a master 
coded video of a coaching meeting. Coders at each site had 
to achieve 80 % reliability on each code prior to coding 
any sessions. Coaching integrity was collected across 
coaches and across practice modules. Sessions were ran- 
domly selected to sample all modules with coaches having 
approximately two sessions per teacher observed 
(M = 1.9, range 1-3). 

For the BEST in CLASS Coaching Integrity form, 
interobserver agreement was collected for 23.8 % of 
observations by a secondary observer and was calculated 
by the number of agreements divided by the number of 
agreements plus disagreements multiplied by 100. Agree- 
ments ranged from 87 to 100 % across items. 

Teacher Implementation 

The BEST in CLASS Adherence and Competence Scale 
(BiCACS; Sutherland et ah, 2014) was developed as a 
treatment integrity measure to assess the adherence and 
competence of delivery of BEST in CLASS practices by 
teachers. Both subscales assessed seven items (Rules, 
Instructional Pace, Precorrection, Opportunities to 
Respond, Behavior Specific Praise, Corrective Feedback, 
Instructive Feedback) using a seven-point Likert-type scale 
ranging from “Not At All’’ to “Very Extensive” for the 
Adherence Scale and “Very Poor” to “Excellent” on the 
Competence Scale. Coaches and research assistants (who 
conducted interobserver agreement observations as well as 
observations in comparison classrooms) participated in a 
2-h training in the administration and scoring of the BiC- 
ACS. Additionally, coaches and research assistants used a 
manual to facilitate scoring in the field. The BiCACS was 
completed for each teacher-focal child dyad following 
direct observations at baseline (before teacher training and 
coaching began), six times during the 14 weeks of the 
implementation of the intervention, and 1 month after the 
implementation of the intervention ended to evaluate the 
adherence and competence of the teacher’s implementation 
of the practices with each focal child in both the BEST in 
CLASS and comparison conditions. 

In order to assess interobserver agreement, a secondary 
observer collected data on 197 of 824 total observations 
across both conditions (23.9 %) and reliability was calcu- 
lated using intraclass correlation coefficients (ICC). For the 
Adherence subscale, using Cicchetti’s (1994) criteria item 
ICCs ranged from “fair” to “excellent” (ICCs ranged from 
.40 to .83 [M = .71, SD = .13]), while for the Competence 
subscale item ICCs ranged from “poor” to “good” (ICCs 
ranged from .12 to .62 [M = .37, SD = .15]). The internal 
consistency of the Adherence and Competence subscales 
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was acceptable (a = 0.825, 0.837 for Adherence and 
Competence, respectively). 

Intervention Acceptability 

The BEST in CLASS Intervention Acceptability Scale was 
completed by teachers in the BEST in CLASS condition at 
posttest. Using a Likert-type scale (1 = low , 5 = high) 
teachers rated 12 items associated with the acceptability of 
BEST in CLASS intervention, such as comfort level with 
implementation, time involved with implementation, and 
fit within their classroom routine. Teachers also rated 
whether they will continue to use the BEST in CLASS 
strategies as well as an overall rating of the intervention. 

Design 

Data for the current study are from year two of an 
ongoing 4-year randomized controlled trial to examine the 
efficacy of the BEST in CLASS intervention model. Lor 
the RCT, teachers were randomly assigned to treatment or 
comparison groups within each school or early childhood 
center. Since this is a preliminary study and we were 
interested in teachers’ implementation of BEST in 
CLASS, an a priori decision was made to conduct follow- 
up analyses on teacher implementation data. Lor the 
present study, descriptive data are presented on the 
coaching components and a repeated measures analysis of 
variance was used to examine teacher implementation 
across the BEST in CLASS and comparison groups. 
These data are reported by phase, whereby means were 
computed across observations at pre-treatment (baseline), 
post-treatment (end of intervention), and maintenance 
(1 month after coaching ended). 

Results 

Coaching Dosage 

Across the 23 participating BEST in CLASS teachers, a 
total of 321 coaching meetings occurred across the 
14 weeks. Teachers received an average of 6.42 h 
(SD = 0.92, Range 5.0-8.73) of coaching meetings across 
the course of the 14- week intervention, with an average of 
13.96 (SD = 0.21, Range 13-14) coaching meetings 
across the 14 weeks. Coaching meetings lasted an average 
of 27.70 (SD = 3.83, Range 1 0—60) minutes and teachers 
received follow-ups (email, phone, or in-person visit) an 
average of 1.84 times per week (SD = 0.48, Range 0-4). 
Modeling and/or prompting occurred an average of 5.97 
(SD = 2.10, Range 3-12) minutes per week. 



Fig. 2 Adherence rating means by phase 



Timepoint 

Fig. 3 Competence rating means by phase 


Coaching Integrity 

Data from coach self-reports indicated that teachers 
received either modeling or prompting or both in 98.97 % 
of sessions. Modeling was used during 79.26 % of ses- 
sions, while prompting was used during 43.87 % of ses- 
sions. Coaching observations consisted of direct 
observation (100 % of sessions), video data collection 
(96.96 % of sessions), and anecdotal data collection 
(99.67 % of sessions). 

Observers coded 42 randomly selected live (47.6 %) or 
videotaped coaching meetings across all coaches (Range 
2-5 observations per coach) using the BEST in CLASS 
Coaching Integrity form. Data indicated that coaches dis- 
played desired collaborative competencies somewhat to 
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Fig. 4 Effect of coaching on teacher implementation of corrective 
feedback — adherence 


extensively in the areas of rapport building (M = 2.95, 
SD = 0.22), communication skills (M = 2.57, 
SD = 0.50), and facilitating collaboration (M = 2.57, 
SD = 0.55). Across observations coaches implemented an 
average of 91.6 % (SD = 13.8, Range 45-100 %) of the 
1 1 procedural items. 

Implementation Integrity 

Data indicated that scores varied on the Adherence sub- 
scale across treatment phase by group, F{ 2, 91) = 40.08, 
p < .001. Post hoc analyses indicated that Adherence rat- 
ings were significantly different between groups at pre- 
treatment, favoring the treatment group (f( 126) = 2.36, 
p < .05, ES = 0.40). Significant differences were also 
found favoring the treatment group at both post-treatment 
(f(101) = 10.26, p < .001, ES = 2.07) and maintenance 
(f(94) = 8.69, p < .001, ES = 1.81). Data also indicated 
that scores did not vary on the Competence subscale across 
treatment phase by group, F( 2,89) = 1.36, p = .26, 
although post hoc analyses did indicate significant differ- 
ences favoring the treatment group at post-treatment 
(t(101) = 2.09, p < .05, ES = 0.41). Figures 2 and 3 show 
a graphic representation of Adherence and Competence 
subscale means across time by group. 

Finally, in order to illustrate the additive effect of 
coaching on teacher implementation of an example of a 
BEST in CLASS practice. Figs. 4 and 5 show a graphic 
representation of teacher adherence and competence of 
delivery means of corrective feedback across eight data 
collection time points for teachers in both BEST in CLASS 
and comparison groups. Adherence data suggest that 
teachers in the BEST in CLASS group provided corrective 
feedback rarely, even after training, but increased their 
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Timepoint 

Fig. 5 Effect of coaching on teacher implementation of corrective 
feedback — competence 


delivery of this practice after coaching on using corrective 
feedback began. In addition, competence data suggest that 
when teachers in the BEST in CLASS group used correc- 
tive feedback prior to coaching, on average it was delivered 
in the “acceptable” range. Following coaching, teachers 
delivered corrective feedback on average within the 
“good” range. It should also be noted that following 
coaching on these practices teacher implementation 
remained higher, even at 1 -month maintenance, than all 
data points prior to coaching. 

Intervention Acceptability 

Teachers reported feeling very comfortable with the 
intervention and training they received and reported that it 
was minimally difficult to implement and minimally dis- 
ruptive to their classroom. Overall teachers rated BEST in 
CLASS highly and reported that they would continue to 
use the intervention. See Table 3 for a summary of findings 
related to intervention acceptability. 

Discussion 

The purpose of this article was to describe the BEST in 
CLASS coaching model and present preliminary data on 
the implementation of the model with teachers in early 
childhood programs serving young children at risk for 
EBD. Findings on the implementation of the coaching 
component of the BEST in CLASS intervention suggest 
that coaches implemented the coaching model with integ- 
rity, and dosage data indicate that teachers received 
approximately 30 min of coaching during weekly coaching 
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Table 3 BEST in CLASS Teacher Acceptability Scale 


Items 

M (SD) 

How comfortable were you with implementing the 
BEST in CLASS intervention? 

4.61 (0.58) 

How time intensive was it for you to implement the 
BEST in CLASS intervention? 

2.78 (1.13) 

How difficult was it for you to implement the BEST in 
CLASS intervention? 

1.70 (0.93) 

How disruptive was it to your classroom teaching and 
routine to implement the BEST in CLASS 
intervention? 

1.87 (0.87) 

How comfortable were you with the amount of training 
you received in gaining competence to implement the 
BEST in CLASS intervention? 

4.61 (0.58) 

How useful was the BEST in CLASS intervention in 
improving the classroom atmosphere? 

4.17 (0.83) 

How useful was the BEST in CLASS intervention in 
improving the focal child/children’s challenging 
behaviors? 

4.09 (0.60) 

To what extent do you think there might be 
disadvantages in using the BEST in CLASS 
intervention? (reverse scoring for this item) 

4.22 (0.80) 

Does the BEST in CLASS intervention include new or 
different strategies that you have not used previously? 

2.78 (1.31) 

Does the BEST in CLASS intervention include familiar 
strategies that you have learned how to implement 
better or more effectively? 

4.45 (0.74) 

Will you continue to use the BEST in CLASS 
intervention in your classroom? 

4.83 (0.39) 

Overall how would you rate the BEST in CLASS 
intervention? 

4.57 (0.59) 


meetings. Teacher implementation data indicated that 
teachers in the BEST in CLASS intervention group expe- 
rienced significant increases across time in their adherence 
scores; teachers in the intervention group also had signifi- 
cantly higher competence scores at posttest than teachers in 
the comparison group, but these differences were not 
present at maintenance. Finally, teachers reported that the 
intervention was not difficult to implement or disruptive to 
their classroom and overall the teachers rated BEST in 
CLASS highly. 

Descriptive data from the current study indicate that for 
approximately 30 min per week during coaching meetings, 
coaches provided teachers with direct coaching via a 
variety of strategies, predominantly modeling, prompting, 
and performance-based feedback (e.g., direct observational 
data, videotaped examples, and anecdotal data) based upon 
observational and anecdotal data that coaches collected 
during their weekly observations in classrooms. In addi- 
tion, teachers received an average of almost two follow-ups 
from coaches per week. Coaching integrity data suggest 
high adherence to the coaching protocol across items as 
well as collaboration competencies associated with the 


coaching protocol. Thus, the BEST in CLASS coaching 
model appears feasible to implement within a randomized 
controlled trial. What requires further study is how feasible 
it is under less controlled and more naturalistic circum- 
stances, for example, with early childhood program staff 
serving as coaches. 

This study adds to a growing literature base supporting 
the use of practice-based coaching to improve instructional 
practices in early childhood classrooms (e.g., Conroy, 
Sutherland, Algina, et al., 2014; Fox et al., 2011; Snyder, 
Hemmeter, et al., 2011) and also contributes to the litera- 
ture on treatment integrity within response to intervention 
models within schools (Burns, Jacob, & Wagner, 2008; 
Sanetti & Kratochwill, 2009). This study provides pre- 
liminary support for practice-based coaching within Tier-2 
intervention models, such as BEST in CLASS, whereby 
coaches provide support for teachers’ implementation of 
evidence-based instructional practices with young children 
with chronic problem behavior. As coaching models 
become more prevalent, the assessment of integrity for 
both the teacher and coach in tiered models of support may 
help increase our understanding of whether models are 
being implemented with high quality, therefore increasing 
the effectiveness of tiered models of support. 

As the number of young children with chronic problem 
behavior in early childhood settings continues to rise 
(Brauner & Stephens, 2006; Carter et al., 2010; McCabe & 
Frede, 2007), so too does the need for feasible and sus- 
tainable evidence-based programs to be implemented by 
teachers in authentic early childhood settings (Domitro- 
vich, Gest, Jones, Gill, & DeRousie, 2010). Results from 
this investigation support the assertion by Joyce and 
Showers (2002) that, when used in combination with other 
professional development activities, coaching can enhance 
the use of practices by teachers. Teacher implementation 
data from the current study suggest that teachers in BEST 
in CLASS classrooms delivered significantly more (i.e., 
adherence) of the BEST in CLASS practices across time 
than did teachers in the comparison group, while also 
improving the quality of delivery (i.e., competence) of the 
practices at the post-treatment time point. It must be 
pointed out, however, that the nature of the treatment 
integrity data collection procedures may have underesti- 
mated the quality of teacher implementation in this study, 
at least in relation to group differences. To illustrate, 
observers rate each adherence item on a seven-point scale 
that assesses the extensiveness (e.g., thoroughness; fre- 
quency) of delivery of specific practices that comprise 
BEST in CLASS. Observers then rate the competence of 
delivery of each observed practice delivered by teachers 
and make a composite assessment of the observation ses- 
sion on a seven-point scale. Therefore, Teacher A can 
provide a large number (e.g., 30) of opportunities to 
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respond (OTR) in a high-quality manner, for example, and 
receive an adherence rating of “7” and a competence rating 
of “6.” Teacher B may provide a small number of OTR 
(e.g., 5), but with high quality as well, and receive an 
adherence rating of “2” and a competence rating of “6.” 
While Teacher A and B have the same competence rating 
for the observation. Teacher A delivered six times as many 
OTR as did Teacher B, highlighting the important link 
between adherence and competence. Therefore, the sig- 
nificant increase in adherence for teachers in the BEST in 
CLASS group, coupled with the small, but not significant, 
increase in competence, suggests that teachers in BEST in 
CLASS were delivering more of the BEST in CLASS 
practices than teachers in the comparison group, and at a 
minimum delivering them just as well. While the rela- 
tionship between adherence, competence, and child out- 
comes remains largely unknown (Harn, Parisi, & 
Stoolmiller, 2013; Sutherland, McLeod, Conroy, & Cox, 
2013), the measurement of adherence and competence in 
the BEST in CLASS efficacy study will allow us to ulti- 
mately compare the effects of these different dimensions of 
teacher implementation on child outcomes (Durlak & Du- 
pre, 2008). 

While future work will examine the relationship 
between these different dimensions of treatment integrity 
and child outcomes, the measurement of both adherence 
and competence of teacher delivery is an important direc- 
tion for the held. In general, the measurement of compe- 
tence of delivery of intervention components has been 
lacking in the education and prevention fields (Harn et ah, 
2013; Sanetti, Gritter, & Dobey, 2011; Sutherland et al., 
2013). Data from the current study suggest that teacher 
competence of delivery of BEST in CLASS practices did 
improve in BEST in CLASS classrooms at posttest, rep- 
resenting an important aspect of integrity which appears to 
be amenable to improvement through methods such as 
coaching. Whereas treatment adherence focuses on whe- 
ther, and how much , a teacher delivers specific practices, 
competence assesses whether a teacher knows when and 
how to deliver an intervention for maximum impact (Bar- 
ber, Sharpless, Klostermann, & McCarthy, 2007). There- 
fore, the competent delivery of BEST in CLASS practices 
requires a teacher to adapt specific components to meet the 
unique characteristics of the classroom and the individual 
child. While teacher competence is hypothesized to play an 
instrumental role in intervention effectiveness (Harn et al., 
2013; Sutherland et al., 2013), its relationship to child 
outcomes remains largely unknown and needs further 
investigation (e.g., Durlak, 2010). 

While it must be kept in mind that this study did not 
examine the additive effects of practice-based coaching to 
training alone, BEST in CLASS is a “value-added” model in 
that many of the practices teachers are trained and coached 


on are naturally occurring in early childhood classrooms 
(Conroy, Sutherland, Algina, et al., 2014). While Figs. 2 and 
3 represent adherence and competence means across all 
BEST in CLASS strategies at three time points, Figs. 4 and 5 
attempt to provide a more fine-grained illustration of teacher 
implementation data for one practice (corrective feedback) 
at eight different time points. These data suggest that 
coaching had an effect on teachers’ adherence (extensive- 
ness of delivery) and competence (quality of delivery) above 
and beyond that from the training alone; specifically, a slight 
decreasing trend was noted for teacher adherence of cor- 
rective feedback prior to the introduction of coaching, with a 
strong latency of change noted for this dimension immedi- 
ately following coaching. Figure 5 shows a decreasing trend 
in competence for teachers in the comparison group prior to 
coaching and again the end of the intervention, while 
teachers in the BEST in CLASS group indicated a strong 
latency of change as well as maintenance of competent 
delivery across time of corrective feedback. Coaching dos- 
age data indicating that teachers received 30 min of 
coaching and less than two follow-ups per week, in con- 
junction with the acceptability data, suggest that the BEST 
in CLASS coaching model is a promising tool for improving 
teacher, and ultimately, child, behavior with a relatively 
small dosage. 

Implications for Practice and Research 

Coaching on interventions such as BEST in CLASS rep- 
resents a meaningful option for early childhood program- 
based personnel to create a more sustainable mechanism 
for behavioral support for focal children at risk for EBD 
within early childhood programs. Specifically, with 
coaching on a Tier-2 intervention such as BEST in CLASS, 
teachers demonstrate the ability to reduce problem 
behavior and improve outcomes for children needing 
individualized behavioral support above and beyond what 
is provided through a typical high-quality early childhood 
curriculum (Conroy, Sutherland, Algina, et al.. 2014). If 
teachers were able to attain and sustain this level of com- 
petence in reducing problem behavior, then they would 
require less assistance for this population of children, 
allowing those support personnel (e.g., early childhood 
administrators; mental health professionals) to focus their 
efforts on the children needing the most intensive levels of 
support and intervention (e.g.. Tier 3). In this manner, 
teachers are better able to individualize instructional needs 
of children, realizing the promise of practice-based 
coaching within tiered-levels of support. 

While practice-based coaching, such as the model used 
in BEST in CLASS, appears to have promise at improving 
the quantity and quality of teachers’ use of evidence-based 
instructional practices, the sustainability of the model 
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remains unknown. Future work should examine ways to 
provide coaching in cost-effective and feasible ways, while 
not jeopardizing the efficacy of the model used in BEST in 
CLASS. Future research may examine the use of on-site 
personnel (e.g., early childhood program administrators, 
mental health professionals, peer coaches) to provide 
coaching. In addition, online models of training (e.g., 
Mitchem et al., 2009; Miller, Fitzgerald, Koury, Mitchem, 
& Hollingsead, 2007) hold promise for both sustainability 
as well as reaching larger numbers of programs, class- 
rooms, and teachers. Future work should explore similar 
online models to provide coaching supports to teachers 
implementing evidence-based practices in their classrooms. 
Fortunately coaching models, such as that used in the 
current study, provide some guidance for researchers 
developing other formats of coaching support. 

Limitations 

Several limitations of the current study should be kept in 
mind. First, as previously mentioned, the current study is a 
descriptive study of the coaching components included in 
the BEST in CLASS intervention and did not examine the 
additive effects of practice-based coaching to training 
without coaching. While the illustrative data across cor- 
rective feedback provide some support for the additive 
effect of coaching, future work should examine differences 
in teacher implementation between those that receive 
training and coaching and those that receive training alone. 
Second, the data from the current study describe the 
implementation of BEST in CLASS, a manualized Tier-2 
intervention model, on a small number of teachers. As 
such, findings may not generalize to other teachers or other 
classroom-based intervention programs, Tier-2 or other- 
wise. Third, teachers may have varying degrees of under- 
standing of behavioral interventions for young children, 
and this knowledge may impact their ability to implement 
programs such as BEST in CLASS. We did not collect data 
on their knowledge of behavioral interventions; therefore, 
we do not know whether BEST in CLASS has differential 
effects across teachers with varying knowledge of behav- 
ioral interventions. Fourth, this study did not examine the 
effects of the BEST in CLASS practice-based coaching 
model on child outcomes. Therefore, future research 
should focus on systematically examining the relationship 
between coaching (e.g., dosage; coaching integrity), tea- 
cher implementation, and child outcomes. Fifth, only 42 of 
321 coaching meetings (13.1 %) were observed to assess 
coaching integrity and data reported here may not be rep- 
resentative of all coaching meetings; as such, readers 
should be cautious in their interpretations of these data. 
Sixth, item level ICCs on the competence subscale ranged 
from “poor” to “good” and as such should be interpreted 


with caution. While this dimension of treatment integrity 
has been understudied in the literature (Hagermoser 
Sanetti, Dobey, & Gritter, 2012; Ham et al., 2013), pre- 
vious work does suggest that competence may be harder to 
code than adherence (e.g., Hogue et al., 2008). Assuming 
competence matters in the delivery of school-based pro- 
gramming (see Goncy, Sutherland, Farrell, Sullivan, & 
Doyle, in press), future research should focus upon 
increasing the reliability of coding of competence items. 
Finally, the BEST in CLASS coaching component is based 
on an expert coaching model. While the ideal iteration of 
such a model involves coaching provided by professionals 
with expertise in coaching and young children with chronic 
problem behavior, further application of the coaching 
model utilizing other formats (e.g., web-based, site per- 
sonnel) remain to be investigated. 

Conclusions 

Improving the outcomes of young children continues to 
gain attention, and increasingly there is a focus on the 
quality of early childhood programs. Teachers need quality 
training and support, both prior to and after they are in the 
classroom, and these needs become even greater as they 
attempt to provide high-quality programming for increas- 
ing numbers of young children with chronic problem 
behavior. Practice-based coaching, like the model descri- 
bed in this paper, appears to be a promising tool to help 
support early childhood teachers in their implementation of 
evidence-based practices in their classrooms. While much 
work remains to be done to maximize the effectiveness of 
such models, the current paper adds additional support to 
the growing literature on practice-based coaching and the 
potential for implementation of such coaching models in 
early childhood classrooms. 
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